Introduction to Multiview Rank Conditions and their Applications : A Review . ∗

Understanding the representations of 3D scenes as encoded in multiple views taken by a camera from different vantage points is central to many tasks in image and video analysis. These tasks range from recovering the camera motion, 3D structure of the scene and detection and characterization of multiple motions in video. We will demonstrate that the natural representations of a 3D scene in 2D images is in terms of the incidence relations among different geometric primitives, which can be concisely characterized by rank conditions of multiview matrices. The proposed rank conditions capture all existing independent multilinear constraints and enable truly global geometric analysis of the multiple views comprised of different geometric features. In addition to the analysis, we present natural factorization based linear algorithms for structure and motion recovery, image transfer and matching across multiple views applicable in both calibrated and uncalibrated setting. We will demonstrate the approach experimentally on a problem of multiframe structure and motion recovery using point and line features and their incidence relations.